1. Field of the Invention
The present invention relates to image processing apparatuses, image processing methods, recording media, and programs, and more particularly, to an image processing apparatus, an image processing method, a recording medium, and a program, suited to cases in which images are output in a predetermined format.
2. Description of the Related Art
In digital cameras (digital still cameras, digital video cameras, and digital cameras which can capture still pictures and moving pictures), liquid-crystal viewfinders are used in stead of optical viewfinders to determine images to be captured. The digital cameras are provided with a video output terminal for outputting image data recorded inside the cameras to an external display unit, such as a television receiver, to display captured images.
Video output formats include the format of the National Television System Committee (NTSC) system and the format of the Phase Alternating Line (PAL) system. The NTSC system is a color television signal system standardized in 1953, has 525 (60 fields) scanning lines, a frame rate of 29.97 frames/second, and an aspect ratio of 3:4, and is mainly used in the United States of America and Japan. The PAL system is a color television signal system proposed in 1962 in West Germany, has 625 (50 fields) scanning lines, a frame rate of 25 frames/second, and an aspect ratio of 3:4, and is mainly used in Western European countries except France.
Since the whole processing of conventional digital cameras is driven by a reference clock signal conforming to the frame rate of the NTSC or PAL system according to the video output format, images in the NTSC or PAL system are displayed on the liquid-crystal viewfinders.
FIG. 1 is a block diagram showing the structure of a conventional digital camera 1. In the figure, solid lines indicate image signal transfer, dotted lines indicate control-signal transfer, one-dot chain lines indicate code-signal transfer, and two-dot chain lines indicate synchronization-signal transfer.
An image signal captured by a charge coupled device (CCD) camera section 11 is converted to a digital signal by an A/D conversion section 12, and input to a signal processing section 13. The signal processing section 13 applies processing, such as gamma processing and processing for converting an RGB signal to Y/Cb/Cr signals, to the input signal, and sends processed image data to a memory controller 14.
The memory controller 14 writes the input image data into a memory 19 through a bus 16, and sends image data written into the memory 19, through the bus 16 to an image compression processing section 17 or a resolution conversion processing section 18 for processing and records processed data into the memory 19, under the control of the central processing unit (CPU) 15. The memory controller 14 also sends image data processed by the signal processing section 13 or image data processed by the image compression processing section 17 or the resolution conversion processing section 18, if necessary, and recorded in the memory 19 to a display-signal processing section 20 through the bus 16, and reads the image data through the bus 16 and outputs to a drive 23.
The CPU 15 controls the operation of the digital camera 1. The CPU 15 receives a reference timing signal from a synchronization-signal generation section 31, described later, and generates control signals for controlling the operations of the signal processing section 13, the memory controller 14, and the display-signal processing section 20 and outputs them.
The image compression processing section 17 applies compression encoding (software encoding) to received image data according to a predetermined compression encoding method, such as JPEG. The resolution conversion processing section 18 applies horizontal or vertical image-resolution conversion processing to received data.
The display-signal processing section 20 receives image data recorded in the memory 19, through the bus 16, and converts the image data to a signal suited when it is output to a display section 21 and displayed, or to a signal suited when it is output to an external unit through a video output interface 22, according to a reference signal generated by a synchronization-signal generation section 31 disposed inside the display-signal processing section 20. The reference signal generated by the synchronization-signal generation section 31 is sent to the CCD camera section 11, to the signal processing section 13, and to the CPU 15.
The display section 21 is formed, for example, of a liquid crystal panel, and displays image data having a predetermined display format (such as that of the NTSC or PAL system), sent from the display-signal processing section 20. The video-output interface 22 is connected to an external unit by a predetermined interface form, and outputs to the external unit, image data having a predetermined display format (such as that of the NTSC or PAL system), sent from the display-signal processing section 20. The video-output interface 22 may be connected to the external unit by wire or by radio.
The memory controller 14 is connected to the drive 23, if necessary. A magnetic disk 41, an optical disk 42, a magneto-optical disk 43, or a semiconductor memory 44 is mounted to the drive 23. By the processing of the memory controller 14, for example, an image compressed by the image compression processing section 17 is sent and recorded in one of the above-described recording media.
In image capturing, a signal input from the CCD camera section 11 is converted to a digital signal by the A/D conversion section 12, and input to the signal processing section 13. The signal processing section 13 applies processing, such as gamma processing and processing for converting an RGB signal to Y/Cb/Cr signals, to the input signal, and sends processed image data to the memory controller 14. When the image size is changed, the image data is sent to the resolution conversion processing section 18 through the bus 16 by the processing of the memory controller 14, horizontal and vertical image conversion is executed, and converted data is recorded in the memory 19 through the bus 16 by the processing of the memory controller 14.
When the image data is recorded in a recording medium mounted to the drive 23, the image data is sent to the image compression processing section 17 through the bus 16 by the processing of the memory controller 14, and compressed by a predetermined compression encoding method, such as JPEG. The compressed image data is output to the drive 23 and recorded in the recording medium, such as the magnetic disk 41, the optical disk 42, the magneto-optical disk 43, or a semiconductor memory 44, by the processing of the memory controller 14.
When moving pictures are captured, the image signal captured by the CCD camera section 11 and processed by the A/D conversion section 12 and the signal processing section 13 is recorded in the memory 19 in units of frames by the processing of the memory controller 14. The memory controller 14 is controlled by the CPU 15, and sends the image signal recorded in the memory 19 to the image compression processing section 17. The image compression processing section 17 compresses the received image signal according to the frame-rate specification (for example, 15 frames/second or 16.6 frames/second) of the moving pictures. More specifically, the image compression processing section 17 compresses the received image signal to images each having 66.6 milliseconds when the frame-rate specification is 15 frames/second, or compresses the received image signal to images each having 60 milliseconds when the frame-rate specification is 16.6 frames/second. The compressed image data is written into the memory 19 or a recording medium mounted to the drive 23, by the processing of the memory controller 14.
The timing reference used in the above-described processing is a vertical synchronization signal VD generated by the synchronization-signal generation section 31 of the display-signal processing section 20. This signal is input to the CPU 15 and to the signal processing section 13, and serves as the reference for necessary signal processing and the processing of the CPU 15.
Conventionally, this reference timing (vertical synchronization signal VD) is generated based on the frame rate of the format (for example, the format of the NTSC or PAL system) of the video signal output from the video output interface.
In many cases, the liquid-crystal panel of the display section 21 used for the liquid-crystal viewfinder has 240 scanning lines according to the NTSC system. In digital still cameras which use such a liquid-crystal panel and operate according to the PAL system, the scanning lines are thinned out to display images on the liquid-crystal panel.
When PAL images are displayed on a liquid-crystal panel having scanning lines according to the NTSC system, since the scanning lines are thinned out, the image quality deteriorates. In such a case, if on-screen-display (OSD) character information is included in a display content, a part of characters may be missing due to thinning out of the scanning lines.
As the performance of CCDs, image capturing devices, has advanced these days, it is demanded that a frame rate and image quality be improved when moving pictures are displayed. As described above, in the conventional digital camera 1, the vertical synchronization signal VD generated according to the format of a video output serves as the processing reference. Therefore, the CCD camera section 11, the signal processing section 13, and the CPU 15 operate at timing based on the NTSC or PAL system according to the format of the video output.
When the processing reference is based on the NTSC or PAL system, however, it is not preferred that the two formats be used in terms of commercial products. In addition, a need to process image data having different frame rates when moving pictures are captured should be avoided. Therefore, in conventional digital cameras, image processing is performed at a PAL frame rate of 25 frames/second, which is the lower frame rate. Even in NTSC operations, images are processed at a frame rate of 25 frames/second. In other words, a high frame rate cannot be implemented.
When a compressed file of a moving picture is generated by software by the processing of the image compression processing section 17, it is difficult to synchronize images with sound at a frequency of 29.97 Hz based on the NTSC system. Therefore, when the image compression processing section 17 actually performs encoding by software, a frame rate (such as 15 frames/second, 16.6 frames/second, or 30 frames/second) which is not based on the NTSC system is used.
When the image compression processing section 17 performs encoding by software, since high compression with the use of motion vectors cannot be executed, the bit rate cannot be largely reduced and high image quality cannot be obtained. When software processing is used for encoding, the same reference timing needs to be used both in capturing mode and reproduction mode.
To capture and record moving pictures with high quality, it is preferred that a special codec integrated circuit (IC) be used for encoding. In these days, many special codec ICs suited to capture and record moving pictures with high quality have been developed and made commercially available.
The already made commercially available codec ICs have been designed so as to operate in the NTSC or PAL system according to the conventional video output. When the codec ICs are used, since the cost increases, systems which allow a method in which software is used for high-frame-rate encoding and a method in which special codec ICs are used for high-image-quality encoding to be both used have been demanded. If both encoding methods can be used in a system, the system can handle either a video camera structured without a codec IC or a video camera structured with a codec IC, according to cost and quality requirements. Specifically, video cameras which can be connected to codec ICs can be structured.